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Inventors 

Liz Sharpe and John Fraser, Scotland 

Applicants 

6S Ltd, Scotland 

Abstract 

Described herein is a system for storing and retrieving media items that comprise the 
group archive of a private group of individuals. When items are stored an index is 



• ass^ciates^eople, time and events with each item; 

• identifies which items constitute "high points"; 

• identifies trails of items that are of special significance when linked in a particular 
order 

Items can then be retrieved by medium type, by particular associations, as predefinet 
trails, as high points or as serendipitous nostalgic sequences. 

Agent 

Piers Kidd, Marks and Clark, London, England. 

US references 



none 



Claims 



We claim: 



1 .6. assoctatjtig each item with: 
• ^ro or more people 

•/ zero or one date, or a period specified by its two end dates 
1 7 /storiwT ° f 6Vent Ch0S6n fr0m the set defined in ste P 1 -2 





2. retrieving items based on one or a combination of: 
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• items with the same medium type 

• items that have been stored as high points 

• stored trails of items 

• items associated with: 

• a particular person or particular people 

• a particular date or time period 

• a particular type of event 

• any combination of the above three (Hereinafter the combination of 
associations is referred to as the focus.) 

• an automated journey of "serendipitous nostalgia", consisting of these 
steps . 

A. establishing a focus 

B. retrieving a number of items within that focus 

shiftmglo a related focus (e.g. same people and time period but 
different event type) 

D. retrieving a number of items within the new focus 

E. repeating steps C and D 

• selecting items at random 

2 recite? 35 ***** 1 ' SteP 1 U f ° r retrieving items in each <* the ways 

3. A method for repeating or sequencing each of the retrieval methods recited in 2. 

4. An automated method and computer-readable medium for maintaining and 
S 1%^?^ ° n b3SiS ° f inpUtS fr0m USerS ' USin 9 the ^2 in 

Background and summary 
Technical field 

Background of the invention 

I^j^T^^^SlSSS^T^f °;° UPS 0f ^P' 6 oiJndMduats to establish 
The scheme is based on the following premises 

' cu^re^ ° f it6mS in different media that are P art ""he group's interaction 
SSStt? *** tiTCan be translated into electronic form 

/linked in the partSlar S * ^ m ° f SpeCial s «ance when 

/ Items can be associated with an^ or all of these 

• a Particular date or time period; 

• a particular type of event. 
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People will want to review the contents of the archive based on one or a 
combination of: 

items of similar medium type; 
high points; 
trails; 

items of similar focus (people, time and event type); 
a journey of serendipitous nostalgia that takes them through items of similar 
focus, then successively shifts to a related focus and reviews items with the 
new focus 
• dipping in "at random". 

Summary of the invention 

The invention is based on establishing an electronic database, storage routines and 
^^^^^T* diff6rent datab3Se P acka 9 es or computer languages can be 

V^^^i w 3 * !! 16 !^ emb I ? rs of a P nVate 9 rou P ™>K together (or an individual works 
independently) to identify, collect, translate or create items in different media e a 
favourite phrases, photographs, verses of poetry, recordings of their own voices dies 
^n,?,T!^f S ' ClipS fr0m their ******** music - These items collectively represent 

nhr^ 9r ° UP ° r ,n ^ idUa '- The items may 3,1 te in the same ^um, e 

photographs or music recordings. y 

I^n^s f^ CidG Whi f ite !? 8 t0 archlve * whether the * «»Wu«e high points for the 

SKJKf T* Q ? t ?- 96ther 10 S6qUenCe t0 create trai,s and ^at the associated 
information is (people, time and events). These details can be created or changed at 
any time subsequent to their inclusion in the database. cnangea at 

At a later time^ individuals in the group or the group as a whole will want to reminisce or 
rev,ew the contents of the archive. This can be done in one or a co^S^ ^T 
ways described in Background of the invention above °™»nanon or tne 
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gr o up' s mult i media i tem 



groups of 
people 



P , 3 . i 




(multjhnedia 
items 



3i~database 
of groups 
and people 



4, i tot i (J- 
table of 
event types 
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Figure 2: The storage process 
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2: 

multimedia 
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12: store high 
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14: store trail 
information 



15: determine 
what associated 
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applies 
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16: store 
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5: calendar* 
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Figure 3: The areais^f the multimedia database 
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Table 2: Storage of high points 



(a 



information 


stored as 


permitted 
values 


necessary? 


multiple 
values? 


notes 


identifier 


whatever the 
database 
package 
allows 


whatever 
the 

database 

package 

allows 


y 


n 


1 


item(s) 


identifier(s) 


those items 
in the 
multimedia 
database 


y 


y 


A high point 
must have at 
least one 
item. 


Table 3: Storage of trails 


information 


stored as 


permitted 
values 


necessary? 


multiple 
values? 


notes 


identifier 


whoever the 
cacRBse 


whatever 
the 

database 

package 

allows 


y 


n 




first item 


identifier 


those items 
in the 
multimedia 
database 


y 


n 


Some 

packages will 
allow multiple 
values to be 
sequenced, 
in which case 
all of the item 
identifiers 
can be stored 
in sequence 
in one field. 


second item 


identifier 


those items 
in the 
multimedia 
database 


y 


n 

- 


A trail must 
have at least 
two items. 
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information 


stored as 


permitted 
values 


necessary? 


multiple 
values? 


notes 


third item 


identifier 


those items 
in the 
multimedia 
database 


n 


n 


A trail may 
have 3 or 
more items. 


further items 


identifier 


those items 
in the 
multimedia 
database 


n 


n 


Typicafly a 
trail will have 
a maximum 
of about 10 
items. 



Table 4: The associated information in the index 



information 


stored as 


permitted 
values 


necessary? 


multiple 
values? 


notes 


identifier 


whatever the 
database 
package 
allows 


whatever 
the 

database 
allows 


y 


n 




medium type 


string 


text, sound, 

image, 

video 


y 


n 




people 


identifier 


those 
people 
stored in the 
database of 
groups and 
people 


n 


y 


could be a 
whole group 


time 


date 


any date 

within 

reasonable 

living 

memory 


n 


y (max 
two 

values) 


if two values, 
denotes that 
item is 
associated 
with a time 
period; future 
dates are 
allowed 


event type 


identifier 


ttx»e 
caputs in 

the event 
table 


n 


n 





Table 5: Some possible combinations for retrieval 



medium 


high 


restrict by 




seren- 




type 


points 


trails 


focus 


dipity 


notes 


S 


X 


X 


X 


X 


only items of particular medium type 






X 


X 


X 


only high points of particular medium 
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X 
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X 






S 
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X 
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all trails 

all trails containing items of particular 
type 
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X 
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X 


X 




✓ 


X 
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X 


X 


X 
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X 
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Figure 4: Sequencing the retrieval processes 
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Figure 5: The generic retrieval process 
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Figure 6: Retrieval by focus 
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Table 6: An example of serendipitous nostalgia 
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Figure 7: Serendipitous nostalgia 



may require interaction 
with the user 



Q begin ^ 



32: identify 
focus 



33: decide how 
many items (n) 
to show with 
same focus 



34: retrieve n 
Items with same 
focus 
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Table 7: Some possible focus shifts in serendipitous nostalgia 



change 


example 


narrow the focus by one level 


if the old focus has no restriction on time period, 
then include a time period in the new focus 


broaden the focus by one level 


if the old focus specifies an event type, let the 
new focus have no restriction on event type 


narrow the people focus 


add more people 


broaden the peopte facul 


remove some of existing people 


narrow the time focus 


reduce the time period from a year to a season, 
from a season to a month, etc. 


make a small focus shift 


make people focus narrower or broader by just 
one person 


make a large focus shift 


select completely new people 


make a larger focus shift 


select completely new people and a different 
event type 


make the largest shift 


select new people, a different time period and a 
different event type 
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Table 8; Some strategies for shifting focus 
strategy 

alternate broadening and narrowing the focus 
broaden and narrow the focus at random 



alternate small focus shifts and targe focus shifts 

small and large focus shifts at random 

rotate focus shifts by people, time and event types 

choose randomly between people, time and events 

make shifts according to observed behaviour of different kinds of groups 



Brief description of the drawings 

Figure 1 shows the main components of the method. 

It is assumed in this figure that users are social groups and that the database of groups 
and people (label 3) already contains records of groups and the people who belong to 
the groups. The group registration process (label 1) is not part of the invention. 

The storage process (label 6) is responsible for associating, with each item to be stored, 
an identifier and any other associated information (labels 3-5) for the index. It also 
stores details of high points and trails. 

The multimedia database (label 7) contains the items themselves, the high points, the 
trails and the index of associated information. 

The retrieval processes (label 8) allow the items to be recovered and reviewed in 
appropriate ways and sequences. 

Table 1 gives a list of event types appropriate for groups of teenage girls. 

Figure 2 is a simple summary flowchart of the storage process, showing the inputs from 
components in Figure 1. It also shows which of the steps require interaction with users. 

Figure 3 shows the main storage areas of the multimedia database: the items 
themselves, the high points, the trails and the index of associated information. 

Table 2 and Tafcte 3 sN#ff1fc irformation that is stored about high points and trails. 

Table 4 shows the W iSMHaiun that is stored in the index of associated information 

Table 5 gives exampJes of how the various retrieval methods can be combined to 
restrict the applicable items for retrieval. 

Figure 4is a simple flowchart of the method for sequencing the various retrieval 
processes. 

Figure Sshows the generic retrieval process for the simple cases of retrieving by 
medium type, high points and trails. Figure 6 shows the special case of retrieving by 
focus. These figures both show where user interaction is required. 

Table 6 shows an example of serendipitous nostalgia and Figure 7 is a simple flowchart 
of the process. 

Table 7 shows different ways for the retrieval process to make individual shifts in focus, 
while Table 8 shows some strategies for making successive focus shifts. 



Detailed description of the preferred embodiment 

(In this description, the label numbers refer to the labels in the preceding figures.) 
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Summary 

The inventors' preferred embodiment is as part of an internet based service that allows 
private groups to maintain and access their electronic media group archives. The 
archives could be single-medium or multimedia. The content comes from the media 
which surround them: films; music; mimicry and sayings (from actors, the group 
members etc.)] TV adverts; TV programmes; books; plays; photographs from personal 
events like dances, school trips, holidays and parties. They would create some of the 
items themselves and obtain some items from online libraries and other sources. 

It is envisaged that the service will be most popular in the first instance with groups of 
teenage girls, women and families, who will maintain and access their group archives 
^ over long periods of time. 

f ""White being very suitable for groups of^enage girls, women and families, the Invention- 
is also expected to be useful for otl^eftypes of private social group, such as workmates, 
sports teams and even individual! Each of these different types of groups might have a 
different set of event types (label 4 in Figure 1) that are relevant to them. Table 1 shows 
a set of event types approprate for teenage girls. 

The embodiment wouldentail many detailed steps, nordescribebn^ common 
practice in software engineering, for checking conditions such as empty lists of 
applicable items. These are omitted in the interests of clarity. 

Registration 

Users of the internet based service register themselves and their groups, indicating who 
are the members of their private group. This information is stored in the register (label 3 
in Figure 1). The register is not part of the invention. In the case of individuals, they 
would be asked to build and maintain their own register of people relevant to their 
archive. 

Users work in their private groups, either physically in the same place or connected by 
remote communication means such as the internet or telephone. They create or collect, 
in a wide variety of ways, media items of interest (label 2 in Figure 1 ) for storage in an 
archive. 

Storage 

The storage process automatically generates a unique identifier for each selected item 
(label 9 in Figure 2) and stores the item (label 10 in Figure 2) in the media items area 
(label 17 in Figure 2) of the multimedia database. The identifier is stored as part of the 
index of associated information (label 20 in Figure 2). The identifier is the first entry in 
Table 4. In addition , the medium type of the item is stored as part of the index of 
associated W&mtation. The tg£££an probably be inferred from the format of the item: if 
not, the user is atted. ThSffWtom type is the second entry in Table 4. 



In some cases it may »#iifcttshed that the items in the archive are all in a single 
medium. In this case it is not necessary to identify or store the medium type. The 
remainder of this description assumes a multimedia archive. 

High points 

Users have the opportunity to state whether the item, either on its own or with other 
items, is a "high point" (label 11 in Figure 2): does it hold some special significance 
e.g. the last day of the group's last year in school; the first date of the quietest member 
of the group. If so. the high point information is stored (label 12 in Figure 2) in the high 
points area of the multimedia database (label 18 in Figure 3). The format in which high 
points are stored is shown in Table 2. 

Trails 

Users can decide that a sequence of items constitutes a special trail that links them in 

some meaningful way For instance: 

1 . an image depicts a still scene from a film 
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2. a sound dip repeats a catch phrase from the film, as spoken by one of the actors 

3. another sound clip repeats the catch phrase in the voice of a school teacher 

4. an image depicts a caricature of the teacher 

The storage process checks (label 13 in Figure 2) to see if users wish to store any trails 
such as that in items 1-4 above and, if so, identifies the sequence of items in the trail. 
This sequence is stored (label 14 in Figure 2) in the trails area of the multimedia 
database (label 19 in Figure 3). The format in which trails are stored is shown in Table 

3. 

An item that is part of a high point can also be part of a trail and vice versa. 
Associated information 

Users are given the opportunity to indicate which, if any, information is to be associated 
with each stored item (label 15 in Figure 2): 

• Which, if any, of the registered people (label 3 in Figure 1 )? 

• Which, if any, of the permitted event types (label 4 in Figure 1 )? 

• Which, if any, date (label 5 in Figure 1)? The date will be either a specific day or a 
range of days, usually in the recent past or the current day. If users decline to 
suggest a date, then the current date can be automatically associated. 

Any associated information thus identified is then added to the index of associated 
information (label 20 in Figure 3). The information that can be stored in the index is 
shown in Table 4. 

High point and trail items can have associated information. 
Building up the archive 

In this way the archive is gradually built up over time, with many media items, some of 
them grouped as high points, some of them forming parts of trails and some of them 
indexed on associated information. 

Retrieval 

At some points in time following storage, members of the group will wish to delve into 
thfe contents of *fte group archive. They might want to do this to have fun, to settle 
arguments, & ffcminisce, to experience a stream of items as a changing backdrop to 
other acttviprih'liie room, to tidy up or to add new items. Whatever the reason, there 
are dtffen^yfcays in which they might want to experience items. 

The bask; methods are: 

1 . experience items of the same medium type or types 

2. experience high points 

3. follow predefined trails 

4. experience items with certain focus 

5. engage in serendipitous nostalgia 

6. dip in at random 

The user can choose the particular method to be used at any time, or the method can 
be determined as in Sequencing the retrieval methods below. 

Focus 

It is assumed that there is always a focus, d etermined either by the user or 
automatically. Th e focus can be: " 

• one or more people 

• a date or a time period 

• an event type 

• one of the above, or a combination of two or all of the above 

The focus can also be "empty", meaning that no person, time or event type has been 
specified. In this case, all items in the database are within focus. 
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Combining the retrieval methods 

Certain combinations of the retrieval methods are feasible, in order to restrict the items 
that are available for retrieval. Examples are given in Table 5. 

Sequencing the retrieval methods 

The six basic methods can be repeated themselves or sequenced with each other to 
form a wide variety of different cumulative experiences. This can be done either under 
automated control or by the user's choice. Figure 4 shows a simple flow chart for 
achieving this. Choosing the retrieval process (label 21 in Figure 4) may be automatic or 
done by the user: users would usually indicate whether they want to control it 
themselves or leave it to the automated process. When done automatically, there is a 
range of possible algorithms, e.g. choose at random; favour methods that have not 
been used recently; favour particular methods for particular types of groups. The figure 
shows a decision box (label 24 in Figure 4} on whether to finish. In practice, the default 
would be to continue once started, until the user interrupts at any point. 

Generic retrieval 

The generic retrieval process for methods 1-3 in the above list is shown in Figure 5. 

The first step is to identify all applicable items in the database (label 25 in Figure 5). In 
methods 1-3 respectively, this means: 

1 . all items of specified medium type or types 

2. all items that constitute high points 

3. all trails 

Next, the process needs to know whether to restrict the applicable items to those that 
are in the current focus (label 26 in Figure 5). If so, all items outside of the focus are 
removed from the list of applicable items (label 27 in Figure 5). 

One of the applicable items must now be selected for retrieval. This can be done either 
by the user selecting from the applicable list, or by automatic selection (labels 28-30 in 
Figure 5). Automatic selection (label 29 in Figure 5) can be done at random, or by a 
more sophisticated method such as giving more priority to items that have not been 
selected recently, giving more priority to items with less recent times associated with 
them, giving priority to items with more specific associated information, etc. 

Finally, for methods 1-3 r*»j»£trveiy the process retrieves (label 31 in Figure 5): 

1. a single item 

2. all items in fl*§8fected high point 

3. all items in sequence in the selected trail 

Retrieval of an item assumes some appropriate way of experiencing it, e.g. playing back 
an audio or video clip. 

Method 4 above — retrieving items with a certain focus — is just a special case of 
generic retrieval, in which: 

• identifying applicable items just entails using the index to identify those items within 



• the question of filtering by focus does not apply (label 26 in Figure 5). 
Figure 6 shows this special case of the retrieval process. 

Serendipitous nostalgia 

Serendipitous nostalgia resembles how one would browse through a box of old 
photographs or personal diaries: a start point may be chosen at random, or on the basis 
of a person, time (period) or event. Reviewing one or more items about that person, 
time or event, a trigger is found in one of the items to a different association, and a new 
sequence of associated items is reviewed. For example, when looking at items relating 
to summer 1 998, the picture of one of the group brings to mind an 1 8 th birthday party in 
which that person made a fool of herself. Finding the diary in which the party is 
described, the focus moves to a different event, also described in the diary, which took 
place around that time... and so the nostalgic trip continues, with apparently random- 
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length sequences focusing on items that are associated in one way, followed by a shift 
in focus and a new sequence of items. 

Table 6 shows an example of a serendipitous nostalgic journey. Columns 1-4 show the 
sequence of retrieved items, while columns 5-7 show the current focus. Whenever the 
focus shifts, it is indicated in column 8. Arrows show how the new focus is generated. 
The journey encounters both a high point and a trail, both of which are shown in the 
table by a grey background. 

Figure 7 shows the serendipitous nostalgia retrieval process. 

The first step is to identify the current focus (label 32 in Figure 7). The user may want to 
change this from the existing focus. 

Then the retrieval process takes over and automatically retrieves a number of items that 
are within focus. The number (n) can be randomly generated or acquired from the user 
or fixed in some other way (label 33 in Figure 7). Once the items have been retrieved 
(label 34 in Figure 7), the figure shows a decision box (label 35 in Figure 7) on whether 
to finish. In practice, the default would be to continue once started, until the user 
interrupts at any point. 

If the decision is to continue, then the focus must be shifted automatically for 
subsequent retrieval (label 36 in Figure 7). There is much scope for a variety of 
methods for individual shifts in focus. Some possibilities are shown in Table 7. There is 
also much scope for a variety of strategies for making successive focus shifts over time 
Some possibilities are shown in Table 8. 

Random retrieval 

Users may also wish to dip into the archive "at random". This may mean literally at 
random l i.e. each item is selected with no reference whatever to the previous one. This 
is easy for the retrieval process to handle. 

It may alternatively mean that they want to see successive items that are as different as 
possible from each other. In this case the retrieval process can ensure the largest 
"distance" between two successive items, based on the associated information of the 
respective items. For instance, items with non-overlapping people are further away than 
those with overlapping people; items with different event types are further away than 
those with the same event types; the difference in times gives a direct measure of item 
distance. 
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RE: Draft UK Patent Application 
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Following your telephone conversation with Tony Luckhurst I now enclose the work so far 
on the speculation and drawings relating to the retrieval side of the invention as a starting 
point for further discuss,on. I do not have a faxable copy of Figure 8 but its content should be 
clear enough from the description. 



Consults! 



Although I will not be involved further, if you have any specific queries, clarifications etc I 
would be happy to receive them. 




Piers B Kidd 
MARKS & CLERK 



Encs 



***>~******* -COm.JOJRN^- **************^*** DATE 04-AUG-2000 ***** T [ME 08=33 *** P. B1 



MODE = MEMORY TRANSMISSION 
FILE NO.= 237 
NO. COM ABBR/NTWK 



001 



INC 



a 



STATION NAME/ 
TELEPHONE NO, 

0131221028? 



START=04-AUG 08: 13 



PAGES PRC NO. PRC 



END=04-AUG 08:33 



017/023 



>W*oK*^**>Mc*)^^ _ 020 ?404 4S10 

MARKS & CLERK 

Intellectual Property 



-MARKSS.CLERK /c ^| Uy\fviO i 



- ***** - 



European Paten I Attorneys 
Chartered Patent Attorneys 
European Trade Mark Attorney; 
Registered Trade Mark At to rne ys 

Established 1S87 



London Office 

R7-G0 Lincoln's Inn Fields 
London WCQA 3LS 

Tei- +<M (0)20 7400 3000 

Fax: +4* {0)20 7404 4910 
£ m ait : e<U nm&marks-clerV r.om 
ntip: //www marks- Clnrk.com 



Ms L Sharpe 
6S Ltd 

10 Learmonth Terrace 
EDINBURGH 
EH4 IPG 



please quote 

our refcrer>ce PBK/ZF 



your reference 
date 



3 August 2000 

Document* 5*6649 



By fax & mail 
No: 0131 221 0287 

Dear Liz 

RE: Draft UK Patent Application 
for Retrieval of Data Items 

Following your telephone conversation with Tony Luckhurst I now enclose the work so far 
on the specification and drawings relating to the retrieval side of the invention as a starting 
point for further discussion. I do not have a taxable copy of Figure 8 but its content should be 
clear enough from the description. 

This is directed to the "effortless" retrieval which is not limited to storage within a database 
on the time/people/events indexing strategy. 

Now that I have actually prepared the two drafts my thinking is that it would probably be 
worthwhile to combine them, at least for a first filing. You will have the opportunity to split 
them apart later if Patent Offices object that there is more than one invention. 

Of course, if there is a psychological benefit to having more patent applications on file then 
we can leave them separate. In some jurisdictions, the United States springs to mind, it might 
be possible to protect the two ideas by way of a single application (the concept of "more than 
one invention" is interpreted a little differently over there). You can consider this at a later 
stage before the anniversary of the first filing date. 
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I have now completed the draft patent specification and I enclose a copy for your comments. 

I have tried to "flavour" the application in the right way to indicate that the archiving and 
retrieval process is based on episodic memory of the group. You will notice at one point in 
the claims I have used the term "predefined socially distinct group" Please consider whether 
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